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Abstract. Many scholars call for more diverse data on learners' behaviour in CALL 
(Colpaert, 2012; Fischer, 2007, 2012; Stockwell, 2012). Student retention and 
the inconclusive efficacy of online language courses (Gaebel, 2013) has received 
increased attention with the proliferation of MOOCs (Koller, Ng, Do & Chen, 
2013). In this paper we introduce initial findings of tracking data over an eight year 
period on forty- three thousand learners’ progression through the seven courses of 
Icelandic Online , an open, guided online course in Icelandic as a second/foreign 
language. Preliminary findings indicate low course completion rates, from 2.4% to 
18.2%, that vary across courses and seem to cluster around certain junctures in the 
course content. Results show that retention varies according to mode of delivery 
(Harlcer & Koutsantoni, 2005), with the blended learning mode (self-directed 
learning on campus with a tutor) as most effective in retaining students, followed 
by the distance learning mode (self-directed learning outside campus with a tutor). 
Retention is lowest among self-directed students learning outside the context 
of formal education. These findings raise questions about factors that motivate 
retention in online courses, which appear to be not only the modes of delivery but 
also the course content. 
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1. Introduction 

Icelandic Online (IOL) is a website that offers seven courses in Icelandic as a 
second/foreign language. The courses are open and free to users. Great effort was 
taken to design ‘plot-driven’ courses, that is, created around storylines related 
to everyday lives of university students, and based on ‘relevant’ SLA principles 
(Arnbjornsdottir, 2004; Chapelle, 1998). The first courses were launched in 2004 
and approximately 140,000 visitors have logged on to the website. Of those, 43,000 
have been active learners on one or more of the courses. The website is maintained 
by the University of Iceland and has had a great impact on access to Icelandic 
(Fridriksdottir, 2015; Hafsteinsson et al, 2013). 

Since 2006, we have been tracking students’ behaviour online. The purpose of the 
tracking is to establish who is using the website, to map out learner’s behaviour 
as they progress through the courses, and to determine the efficacy of the course 
materials, whether the pedagogic principles that guided the course development 
process have led to language acquisition. The study of retention in online courses 
and the inconclusive results regarding their efficacy (completion rates ranging 
from 2 to 10 percent) has received attention recently due to the proliferation of 
Massive Open Online Courses (MOOCs). Koller et al. (2013) have questioned 
whether retention is the right metric by which to measure success in online courses 
and argue that retention should be considered within the context of student intent. 

The effectiveness of different modes of learning has also been a focal point with 
findings showing that the blended learning mode is more effective in terms of 
student retention than the distance learning mode (Harker & Koutsantoni, 2005). 
Fischer (2007) calls for diverse data on students’ actual behaviour while engaged 
in language learning online. Stockwell (2012) and Colpaert (2012) suggest that 
the Computer Assisted Language Learning (CALL) knowledge base may be too 
reliant on 1) researchers studying their own and their students’ use of technology 
and 2) surveys of what learners say they do online. There are fewer studies on 
what learners actually do while studying language through computers (including 
mobile devices). This is one such study. This article describes initial findings from 
an analysis of eight years of collected tracking data from forty-three thousand users 
of Icelandic Online. 

2. Method 

The main objective in this initial phase of the study was to retrieve demographic 
information about users, tabulate overall retention rates for all courses, and to 
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examine specifically the effect of different modes of delivery on retention. Available 
modes were: open courses (without any intervention), distance non-credit bearing 
courses with a tutor, and blended courses where Icelandic Online serves as part of 
face to face credit bearing courses. 

The demographic information is provided by users as they log in to the website. 
The retention data is collected through a tracker that is built into the software 
where the position of a user is tracked in each course by ‘page’ 3 . Each ‘page’ may 
include 3-6 learning objects. If a user takes multiple courses the system tracks user 
progression through all courses. All the data is stored in a database, and transferred 
to an SPSS statistics program for analysis. 

3. Results 

The registration data provide metrics on enrolment, gender ratio, age, educational 
background and geographic distribution for all visitors on Icelandic Online from 
2006 to 2014 (N=l 39,941). Of those, 31% are active learners (A=43,468), that 
is to say, learners who went beyond eight activities in a course. In this study we 
focus exclusively on the active learners. The registration data reveal that 51% of 
Icelandic Online s active learners are female and 49% are male, 69% of the learners 
are under the age of 31 years, 54% of the learners have received a university's 
degree, and 63% come from ten countries of origin 4 , leaving the rest scattered 
around the world. 

The data reveal that the overall completion rates of the seven courses of Icelandic 
Online range from 2.4% to 18.2%. As part of an effort to break down retention 
data and better understand the nature and motivation of the high withdrawal 
rates, two Icelandic Online courses (IOL1 and IOL2) were examined. In the table 
below we compare the three different modes of delivery for completion of IOL1 
and IOL2, showing from 2.9% to 8.3% completion rate in IOL1, and from 4.4% 
to 14.2% completion rate in IOL2. 

In short, the study found significant differences in both courses between the three 
modalities in terms of student retention, with the blended learning mode proving 
to be much more effective in retention than the open learning and the distance 
learning mode. The results are presented in Table 1 . 


3. Page here is a reference to organisation of content and not to an actual page online. 

4. The top ten countries are the US, Germany, Poland, the UK, France, Canada, Sweden, Norway, Spain, and Italy. 
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Table 1. Course completion in three different modes of delivery in Icelandic 
Online 1 and 2 



Icelandic Online 1 


Icelandic Online 2 

Mode of Delivery 

Open 

Distance 

Blended 


Open 

Distance 

Blended 

Completion % 

2.9% 

4% 

8.3% 


4.4% 

4.8% 

14.2% 

N/total 

189/6419 

4/101 

36/434 


152/3462 

3/62 

40/281 


Additionally, the tracking data revealed regular attrition patterns across all modes 
of delivery in both courses. Figure 1 shows the concentrations of drop-outs at 
certain junctures in the course (IOL2). The concentration of drop-outs at these 
particular junctures may have several explanations that will be discussed in the 
next section. 


Figure 1. Attrition pattern in three different modes of delivery in Icelandic 
Online 2 
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4. Discussion 

The results of the tracking data presented above resonate with findings on student 
retention in MOOCs where overall completion rates are low (Gaebel, 2013), and 
attrition rates are highest at the very beginning of courses (Reich, 2014). Our data 
clearly demonstrates that blended learning is most effective in keeping students 
in courses. As shown in Figure 1, in one of the courses examined specifically 
in this study (IOL2) all three modes have large drop-out rates at the beginning 
of the course, on the first three ‘pages’ (112-114). Bundles of drop-outs are not 
as consistent as students advance in the course. Furthermore, Figure 1 presents 
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an interesting perspective in that although the drop-out rates appear in bundles 
at certain junctures, there are also many sections where learners do not leave. 
This applies across modes of delivery. At this point we do not know whether 
these bundles of drop-outs (or retentions) are related to course content, types of 
learning objects/technology, learning objectives, student intent, or other personal 
reasons. 

5. Conclusions 

This study raised more questions than it answered. We are in the beginning stages 
of a mixed method research project. The aim is to gather as much data as possible 
on whether the pedagogical goals and objectives that guided the development 
of Icelandic Online have led to language acquisition. In the next phase of this 
research, the factors influencing learners' decisions to withdraw or persist in a 
course will be explored further. The difference between the blended learning mode 
and the other two modalities in terms of retention as well as the reasons for the 
bundles of drop-out and persistence will be investigated further. This will be the 
focus of the next phase of this study where evidence from students’ self-reports in 
the form of surveys and interviews will be explored. 
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